Container-based Cluster Management Platform for Distributed Computing

Several fields of science have traditionally demanded large-scale workflows support, which requires thousands of CPU cores or more. Since users’ demands for software packages and configuration is the difference, an approach to making available in real time a service environment desired by users without significant challenges for administrators is necessary. In this paper, we present a container based cluster management platform and introduce an implementation case to minimize performance decline and to provide a dynamic distributed computing environment desired by users. This paper makes the following contributions. First, a container based virtualization technology is assimilated with resource and job management system to expand its applicability to support large-scale scientific workflows. Second, an implementation case in which docker and HTCondor are interlocked with each other is introduced. Lastly, docker and native performance comparison using two widely known benchmark tools and Monte-Carlo simulation results implemented using various programming languages